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With advances in available bandwidth from spacecraft and between terrestrial control 
centers, digital motion imagery and video is becoming more practical as a data gathering 
tool for science and engineering, as well as for sharing missions with the public. The digital 
motion imagery and video industry has done a good job of creating standards for 
compression, distribution, and physical interfaces. Compressed data streams can easily be 
transmitted or distributed over radio frequency, internet protocol, and other data networks. 
All of these standards, however, can make sharing video between spacecraft and terrestrial 
control centers a frustrating and complicated task when different standards and protocols 
are used by different agencies. This paper will explore the challenges presented by the 
abundance of motion imagery and video standards, interfaces and protocols with suggestions 
for common formats that could simplify interoperability between spacecraft and ground 
support systems. Real-world examples from the International Space Station will be 
examined. The paper will also discuss recent trends in the development of new video 
compression algorithms, as well likely expanded use of Delay (or Disruption) Tolerant 
Networking nodes. 


I. Introduction 

Imagery has been a part of space flight from the very beginning. Imagery was one way of proving to the world 
the endeavors were real and sparked the imagination of the public. Imagery also became a key tool for science and 
engineering as missions became more numerous and complex. After the safe return of the crew of Apollo 13, 
engineers were anxious to see the pictures taken of the stricken Service Module. Those images helped engineers on 
the ground confirm what caused the near fatal accident. Likewise, a puff of smoke from one of STS-51L’s solid 
rocket boosters, seen from an engineering motion picture camera, pointed investigators to the eventual conclusion 
that one of the booster’s seals led to the destruction of Challenger. A few key frames of high-speed film alerted 
engineers of a foam strike during the launch of STS- 107. When Columbia disintegrated during entry, that foam 
strike became a likely cause of the accident. Accidents and near-misses in spaceflight utilize imagery, often video 
and motion imagery, as a tool to give clues into root causes of failures. What was once seen as a “nice to have” is 
now seen as a critical tool for engineers and spacecraft designers. 

In the modern era, even un-manned missions feature video or motion imagery capabilities. Advances in 
compression and modulation make the addition of imagery capabilities possible. Often times, the application is 
something other than conventional video and is referred to as motion imagery instead. Conventional video refers to 
standard television system compatible applications that can be routed, monitored and recorded with readily available 
television equipment. Motion Imagery can include conventional video, but also refers to applications other than 
standard video, often having unique frame rates, resolutions and algorithms and can be viewed with computer 
screens and traditional television monitors. 

As we look to future missions, manned and unmanned, to the Moon, Lagrange points, Mars, and beyond, it is 
logical to expect some type of motion imagery system will be a feature of the spacecraft. Back on Earth, multiple 
space operations centers will want to monitor, record, archive and redistribute the imagery. It is likely many future 
missions will be multi- Agency International projects, with space operations centers across the globe monitoring 
activities at the same time. 

Technology advancements in a broad variety of fields related to motion imagery are enabling future missions to 
utilize commercial equipment, along with emerging standards, to make motion imagery a practical part of space 
operations. Commercial advancements in cameras and sensors, compression, and modulation are key contributors. 
Meanwhile, the Consultative Committee for Space Data Standards (CCSDS) has working groups publishing 
standards for Internet Protocol over CCSDS packet protocols, Delay (or Disruption) Tolerant Networking, and 
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digital motion imagery for space applications. The era of full motion imagery as part of space communications and 
operations is upon us. 


II. Motion Imagery Operations Concepts 

Motion imagery capabilities on spacecraft suffer from the constraints of bandwidth, power consumption and 
mass. Without these constraints, it is likely that launch control rooms and space operations centers would resemble 
security monitoring centers at casinos with multiple monitors and views of rockets, payloads, rendezvous 
maneuvers, and astronaut activities. 

Rocket-based motion imagery is particularly constrained. High Definition Television (HDTV) cameras output 
nearly 1.5 Gigabits per second uncompressed. Converting that output to a compressed, packetized transport stream 
capable of live distribution to a ground station can be a daunting challenge. Fortunately, commercially available 
encoders are getting more efficient and smaller. The most bandwidth efficient encoding utilizes h.264 (or MPEG-4 
Part 10). The trade-off is the h.264 algorithm compresses not only frames, but also pixel blocks within and between 
multiple frames. Inter- frame algorithms that compress groups of pictures are very efficient, but imagery analysts 
prefer encoding algorithms that utilize intra-frame encoding, where each frame is maintained. Intra-frame encoding 
algorithms, such as Motion JPEG-2000, require considerably more bandwidth for live streaming motion imagery. If 
analysis quality imagery is needed from a rocket based imagery system, it is likely the imagery will need to be 
recorded and retrieved later. One possible solution might be to design a recording device that could be ejected and 
retrieved later. The infamous “ring” shot from the staging on the Saturn V rockets was retrieved this way. A small 
16MM film camera was placed on the second stage. It was ejected shortly after second stage ignition and retrieved 
from the ocean. Another option is to record the imagery on a drive in the spacecraft. Spacecraft usually have more 
communications bandwidth than rockets in flight. When a higher bitrate communication link is established, the 
imagery stored from the rocket-based imagery system can be downloaded to ground systems. However, the 
spacecraft communication link must have a high bit-rate to get all that data to the ground, and will likely be 
interrupted with communications outages before a large file can be downlinked. Applications for interrupted file 
transfers that can resume the transfer at the point of interruption are necessary to make this practical. 

Motion imagery on spacecraft is often used to confirm staging operations and mission events such as solar array 
deployment, rendezvous and docking. These events often must be monitored live, with very low latency. The 
amount of data generated for live digital motion imagery will far exceed the combined requirements for other 
telemetry and voice data coming from the spacecraft. In effect, digital motion imagery will define the overall 
communication link specifications, including bandwidth and jitter specifications. Live streaming video using h.264 
compression could require 6-8 Megabits per second of bandwidth for high quality, compared to all other telemetry 
which might total 2 Megabits per second. Compressed video streams, particularly when using h.264, are highly 
susceptible to packet loss and jitter. A decoder will freeze an image or produce no image at all if the data stream is 
interrupted or packets arrive out of order beyond the decoder buffer’s capacity to reorder. Streaming video from a 
spacecraft will require a communications link system with jitter (packet delay variation) not to exceed 10 ms, and bit 
error rates not to exceed 1 x 10" 6 in order to ensure a live video data stream that can be received and decoded in real- 
time. 

As cameras and compressors improve and available bandwidth to the ground increases, the quality of the motion 
imagery improves. The International Space Station (ISS) can now stream live HDTV to the ground, for example. 
Humans are exploring beyond low-Earth orbit. Live or recorded spacecraft-based motion imagery has been limited 
to low-Earth orbit and the fuzzy black and white television from the Moon landings. As we inevitably move human 
exploration beyond the Moon and continue to send sophisticated spacecraft into the solar system, motion imagery 
systems get more complex. The concept of “live” monitoring changes to perhaps mean “streaming” versus file- 
based transmission of motion imagery. Motion imagery systems for exploration beyond Earth orbit will need to be 
self-aware, capable of filtering through irrelevant content and storing what is considered to be important to be 
distributed later. Space operations centers back on Earth will retrieve files for playback later, or monitor recorded 
content as a stream in “real-time”. 

The good news for designers of motion imagery systems for space applications is there are a variety of 
commercially available technologies to choose from, where little or no new development of these technologies is 
necessary. The bad news is these choices lead to different implementations across multiple spacecraft and space 
centers. One motion imaging system may utilize standard definition quality cameras with interlace scanning, 4:3 
aspect ratio, a frame rate of 25 frames per second, and use MPEG-2 compression. Another might use high definition 
cameras with progressive scanning, 16:9 aspect ratio, a frame rate of 60 frames per second, and use h.264 
compression. A number of format conversions were necessary, for example, for monitoring of the European Space 
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Agency’s Automated Transfer Vehicle maneuvers and docking to the ISS. Space operations centers may require up- 
converters, down-converters, frame rate converters, and other translators in order to support motion imagery for 
multiple missions. Each conversion degrades and modifies the imagery. If analog sources are used somewhere in the 
signal path, the image degradation is even worse. Further, multiple format conversions can make trouble shooting 
very difficult if the final imagery has artifacts or is not what was expected. Narrowing down all these options to a set 
of common applications will make interoperability much easier. The Consultative Committee for Space Data 
Standards Motion Imagery & Applications Working Group http://cwe.ccsds.Org/sis/default.aspx# SIS-MIA is 
developing a set of standards to aid in the selection of these many commercial standards to meet a common set of 
applications. 


III. Enabling Technologies and Strategies 
A. Rocket-based Motion Imagery 

Live, streaming rocket-based motion imagery is typically constrained by S-band telemetry links. The challenge 
for engineers and designers is how to trade the limited bandwidth, often limited to 1 0 Megabits per second, with the 
desire to have high temporal resolutions (high frame rates) while maintaining high spatial resolutions (lines and 
pixels), with multiple camera views. Strategies include taking advantage of knowledge about the mission events 
during ascent so cameras are switched on and off to provide motion imagery only when relevant events are 
occurring, such as ignition, lift-off, roll maneuvers and staging. The system automatically switches from one camera 
view to another during the mission time-line, using timing from a master clock on the rocket or motion sensors to 
trigger the switches. Another strategy is to “window” the motion imagery so that only the relevant information is 
encoded and transmitted (See Figure 1). For example, a High Definition camera has an aspect ratio of 16:9. A 
camera mounted on the side of a rocket, with a view looking down the side toward the engines, for example, might 
have a field-of-view that includes irrelevant information, such as sky or the sides of the rocket nearest the camera. If 
the engineers are mainly interested in the field-of-view near the bottom of the rocket, the system can be designed to 
“window” or ignore the irrelevant parts of the image, compress only the part of the image that is wanted, and route 
that slice of the image for distribution. 

Another strategy might be to vary frame rates during different stages of rocket ascent. During early ascent 
phases, for example, the frame rate from one camera might be set to 60 frames-per-second, then reduce the frame 
rate to 15 frames-per-second for a later phase to allow another camera to utilize bandwidth to distribute motion 
imagery at the same time. Camera “A” might start the ascent phase at 60 frames-per-second until Launch + 30 
seconds, then camera “B” begins operating. Camera “B” may begin at L + 30 seconds at 45 frames-per-second and 
camera “A” is reduced to 1 5 frames-per-second. In this way, two views are available using the same bandwidth. 

The system could also buffer frames to be transmitted later in the ascent phase. Early in the rocket ascent the 
system could transmit 30 frames-per-second while buffering an additional 30 frames-per-second. Late in the ascent 
phase it could transmit the buffered frames. Using timing information encoded within the motion imagery, the full 
60 frames-per-second could be reassembled. This strategy has the advantage of providing at least some motion 
imagery from cameras during ascent in case of catastrophic event, vs. having all motion imagery captured on board 
for full transmission later in flight. An accident might destroy the drive where the imagery was stored, meaning no 
video was ever received. 


3 

American Institute of Aeronautics and Astronautics 




Figure 1. Still frame from rocket-based motion imagery with windowed content. 


B. Spacecraft-based Motion Imagery 

One of the more complex and expensive aspects of an external spacecraft camera system is controlling and 
aiming (if the camera isn’t fixed). A camera is much more useful for space operations if it can tilt and pan and if the 
lens has a variable focal length. In that way controllers can 
change the field-of-view, which is very useful for more complex 
spacecraft like the ISS, but would also be helpful if monitoring 
space walking astronauts on an asteroid, Moon base, or on Mars. 

But such a system requires a command uplink from control 
systems on the ground to the spacecraft, or control systems in the 
spacecraft, or both. The mechanical systems required for panning, 
tilting, and zooming are challenging. The external space 
environment requires special materials, coatings, and lubricants to 
enable a lens to change focal lengths, for example, without 
seizing or allowing the lubricants to leak onto the lens elements. 

A pan-tilt unit is also expensive and complicated to deploy 
externally in the space environment. The extreme temperatures, 
radiation, and, in the case of the Moon, regolith, can damage or 
destroy the mechanisms. A strategy to alleviate these concerns is 
a camera system that has no moving parts. As cameras get 
smaller, while at the same time have improved spatial resolution, 
it is possible to design and deploy a camera system that can have 
a 360° field-of-view by placing multiple cameras next to each 



Figure 2. Multi-camera concept 
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other (see Figure 2) and electronically “stitching” the video together. By using cameras that over-sample the desired 
resolution, the zoom function can be provided by electronically zooming into the sensor. For example, if 1280 x 720 
progressive @ 60 frames-per-second is the desired output, using cameras that have resolutions of 1920 x 1080 or 
greater would allow digital zooming into a 1280 x 720 image space without degrading the image quality. Further 
autonomy can be achieved by adding software that monitors the video from all the cameras and automatically 
switches the field of regard to whatever programmers or engineers decide is of most interest. For example, such a 
camera system on Mars could be designed to be sensitive to motion. Rather than storing or streaming irrelevant data, 
it could detect a brief event such as a “dust-devil”, track the event, tag, store and transmit it when the 
communications link was available. The rest of the time the imagery from all the cameras would be buffered 
temporarily using electronic storage within the camera system and eventually discarded until a relevant event 
occurred. Such a system would be more efficient for both the storage system as well as utilizing precious bandwidth, 
and would not require commanding from the ground or another spacecraft. 

C. Distributing Motion Imagery 

The most complex aspect to deploying robust motion imagery systems on future exploration missions will be 
routing and distributing the streams of data. Imagine a mission to an asteroid, the Moon, or Mars, where multiple 
spacewalking astronauts are deployed. Each astronaut might have a helmet-mounted camera. In addition, cameras 
mounted on the spacecraft or deployed on rovers or stands might serve as third person monitors of the activity. If the 
mission is international, space operations centers around the globe will be monitoring the video feeds (See figure 3). 
If the mission is near-Earth, the feeds will be monitored in real-time, with very low latency from actual time of 
events. If the mission is at an asteroid or on Mars, the video may be arriving tens of minutes behind actual time. 



Figure 3. Multiple sources of video distributed across multiple satellites or “nodes”, with high potential for 
disruption. 


5 

American Institute of Aeronautics and Astronautics 




If the motion imagery is being streamed, it will be subject to communication link integrity. Compressed video 
streams are highly susceptible to jitter and packet loss. Decoders will freeze the image or stop outputting usable 
imagery if the data stream is incomplete or interrupted. If the imagery being streamed is from stored files on the 
spacecraft, it could be resent. If, however, the motion imagery is being streamed in real-time from the spacecraft, 
with no spacecraft recording system, then it is critical that the data stream is robust and protected as much as 
possible. Disruption or Delay Tolerant Networking (DTN) systems would dramatically improve the ability to 
distribute motion imagery data streams. 

DTN should enable a practical solar system internetworking capability. Typical Internet Protocol networking 
utilizes a packet structure that assumes a continuous network link. DTN is designed for networks that are subject to 
disruptions or breaks in the network link. There are three classes of video transmission under DTN: 1) File transfer 
of recorded video; 2) Real-time delivery, best effort; 3) Real-time delivery, no packet drop. 

With DTN, in the case of an imagery file download from Mars, for example, the file will be bundled into larger 
packets and distributed across the various links. If there is a disruption in the link, the file would not have to be 
retrieved again from the source, but rather the missing parts of the file would automatically be forwarded when the 
communications link was restored. The Licklider Transmission Protocol 1 will note loss of DTN bundles (which 
would contain 1 -n video frames) and request retransmission of missing bundles. 

If motion imagery is being streamed across a DTN enabled network, space operations centers could monitor the 
stream being received in real-time with assurance that parts of the stream that may be missing due to disruptions can 
be retrieved or reconstructed later. In some instances, it is likely a space operations center will want an interrupted 
stream to resume “in progress”, resuming in real-time with the latest video to arrive. In that case, the prioritization 
will need to be for real-time packets that arrive to be processed first (best effort). Packets that arrive late or out of 
order would be stored and used to reconstruct the entire sequence after the fact. The data would be forwarded using 
DTN Bundle Protocol 2 without custody transfer and would be available to the receiver/decoder on a best effort 
basis. 

In other instances, a space operations center will want to resume where the stream left off, so that no video or 
motion imagery is missed or incomplete. In that case, prioritization would be for continuity of packets (no packet 
drop). Packets received out of order or later would be buffered until the sequence could be properly reconstructed. 

In either case, the video acquisition system should include timing data within the compressed video data stream 
as an aid to reconstructing the correct sequence order. The Consultative Committee for Space Data Systems has 
standards documents under review that address both DTN and how DTN can be utilized for video applications. See 
http://cwe.ccsds.org/default.aspx for the latest versions of these documents. 

IV. Conclusion 

Video and motion imagery is ubiquitous in modern life here on Earth. It is difficult to go anywhere without 
seeing a television monitor. Modem mobile devices are capable of playing movies and streaming live video. In the 
early days of spaceflight, video and motion imagery was rarely practical. Today, even routine unmanned rocket 
launches feature live streaming video from camera systems attached to the side of the rocket. As bandwidth 
increases, modulation systems evolve, and camera systems and compression all improve, video and motion imagery 
are likely to become requirements for future spacecraft and exploration missions. Engineers and systems designers 
for these future spacecraft will be able to take advantage of smart systems that can isolate relevant information from 
the irrelevant, stitch together motion imagery from a suite of cameras into a single video stream, and distribute those 
streams across a Delay Tolerant Network system to enable robust monitoring and recording of motion imagery at 
multiple space operations centers across the globe. 
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Appendix A 
Acronym List 


CCSDS 

DTN 

HDTV 

ISS 

MPEG 


Consultative Committee for Space Data Standards 
Delay (or Disruption) Tolerant Networking 
High Definition Television 
International Space Station 
Moving Pictures Expert Group 
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Appendix B 
Glossary 


Consultative Committee 
for Space Data Systems 

Delay Tolerant 
Networking 

Motion Imagery 
Video 


Founded in 1982 by the major space agencies of the world. A multi-national 
forum for the development of communications and data systems standards for 
spaceflight. 

An approach to computer network architecture that addresses technical issues 
in networks that lack continuous network connectivity. 

Refers to specialized imaging systems that generate continuous or sequential 
streaming images. Covers applications not considered traditional video. 

Traditional applications of worldwide standardized television applications such 
as NTSC, ATSC, PAL and SEC AM. 
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